Implicit relevance feedback from a multi-step search process: a use of query-logs

We evaluate the use of clickthrough information as implicit relevance feedback in sessions. We employ records of user interactions with a commercial news picture portal: issued queries, clicked images, and purchased content. Our study investigates how much of a session’s search history (if any) should be used in a feedback loop. We assess the benefit of using clicked data as positive tokens of relevance to the task of estimating the probability of an image to be purchased. We find that a short history of past queries helps improve ranking, and that terms derived from clicked documents lead to a much higher effectiveness, while blind relevance feedback is not beneficial for the task. 1 Evidence of user interaction: Query Logs (QL) Logs of queries issued and the subsequent interactions with the query results, briefly referred to as ‘query logs’ (QLs) in this paper, provide a basis to adapt a relevance model to reflect what we have learned about the user’s information need. A set of QLs recorded when subscribers to Belga Picture were searching for images to be purchased online, allows us 1) to investigate how valuable clicks are as source of (implicit) relevance feedback in a multi-step search session and 2) to observe how much search history (if any) may lead to an improvement in the ranking of what we believe to be a determinately relevant document: the picture that a user is known to have purchased at the end of a search session. A QL registers, for each session, three types of user interactions: query submissions (Q), a possibly empty set of clicks (C) on the retrieved results, and, purchases (P); an anonymous identifier labels each step. Previous studies diverge in their findings about how much evidence of user interactions (Q and C) should be used for feedback: Tan et al. report in [5] that long term search history may improve web retrieval, while the authors of [4] argue to emphasize short-term query context. Also, Gong et al. question whether clicked data should be accepted as positive evidence of a document being relevant without a quality 1 A European news agency: http://picture.belga.be/picture-home/index.html, log data collected within the VITALAS project: http://vitalas.ercim.org.